Constrained total undiscounted continuous-time Markov decision processes

摘要

The present paper considers the constrained optimal control problem withtotal undiscounted criteria for a continuous-time Markov decision process(CTMDP) in Borel state and action spaces. Under the standard compactness andcontinuity conditions, we show the existence of an optimal stationary policyout of the class of general nonstationary ones. In the process, we justify thereduction of the CTMDP model to a discrete-time Markov decision process (DTMDP)model based on the studies of the undiscounted occupancy and occupationmeasures. We allow that the controlled process is not necessarily absorbing,and the transition rates are not necessarily separated from zero, and can bearbitrarily unbounded; these features count for the main technical difficultiesin studying undiscounted CTMDP models.